Image capturing apparatus, control method thereof, and non-transitory computer-readable storage medium

ABSTRACT

An image capturing apparatus which a lens unit is interchangeable, comprises an image capturing unit which captures images using an image sensor, an synthesizing unit which synthesizes a plurality of images, and a controlling unit which controls predetermined synthesis processing for creating a synthetic image by causing the synthesizing unit to synthesize a plurality of images that have been obtained by the image capturing unit performing image capturing a plurality of times, wherein, if it is judged that a lens unit that has been replaced after a first image for the synthesis processing was acquired is a lens unit having an image circle that is different from that of a lens unit that was mounted before the replacement, the controlling unit controls the image capturing unit to not capture a new image for creating the synthetic image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional of application Ser. No. 16/448,685,filed Jun. 21, 2019, the entire disclosures of which is herebyincorporated by reference.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an image capturing apparatus, a controlmethod thereof, and a non-transitory computer-readable storage medium.

Description of the Related Art

There are digital cameras that can perform so-called multiple-exposureimaging in which one synthetic image is obtained by synthesizing aplurality of shot images, and the synthetic image is recorded. There aredigital cameras in which the number of images to be synthesized or themethod of synthesis processing such as weighted average, synthesis ofrelatively bright portions, or synthesis of relatively dark portions canbe selected before shooting is performed, and there are also digitalcameras in which an image that has already been shot can be selected asa base image, and a multiplexed image can be created by synthesizing animage to be shot on the base image.

Japanese Patent Laid-Open No. 2012-19343 and Japanese Patent Laid-OpenNo. 2007-221237 disclose a technique for changing settingmid-multiple-exposure imaging. Japanese Patent Laid-Open No. 2012-19343discloses a technique in which the setting of the aspect ratio of animage to be recorded can be changed mid-shooting, in a digital camerathat can perform multiple-exposure imaging. Also, Japanese PatentLaid-Open No. 2007-221237 disclose a technique in which settings such asrecording quality and recording size can be changed mid-shooting, in adigital camera that can perform multiple-exposure imaging.

In an interchangeable lens type camera, the lens unit can be replacedwhile multiple-exposure imaging is being performed. There are varioustypes of replaceable lens units, and it is conceivable that an unnaturalsynthetic image is generated depending on a replaced lens unit.

SUMMARY OF THE INVENTION

The present invention in its first aspect provides an image capturingapparatus which can interchange a lens unit, comprising: an imagecapturing unit which captures images using an image sensor; asynthesizing unit which synthesizes a plurality of images; and acontrolling unit which controls predetermined synthesis processing forcreating a synthetic image by causing the synthesizing unit tosynthesize a plurality of images that have been obtained by the imagecapturing unit performing image capturing a plurality of times, wherein,if it is judged that a lens unit that has been replaced after a firstimage for the synthesis processing was acquired is a lens unit having animage circle that is different from that of a lens unit that was mountedbefore the replacement, the controlling unit controls the imagecapturing unit to not capture a new image for creating the syntheticimage.

The present invention in its second aspect provides an image capturingapparatus which can interchange a lens unit, comprising: an imagecapturing unit which captures images using an image sensor; asynthesizing unit which synthesizes a plurality of images; and acontrolling unit which controls predetermined synthesis processing forcreating a synthetic image by causing the synthesizing unit tosynthesize a plurality of images that have been obtained by the imagecapturing unit performing image capturing a plurality of times, wherein,if it is judged that a lens unit that has been replaced after a firstimage for the synthesis processing was acquired is a lens unit thatnecessitates predetermined image correction, the controlling unitcontrols the image capturing unit to not capture a new image forcreating the synthetic image.

The present invention in its third aspect provides a control method ofan image capturing apparatus which can interchange a lens unit andincludes an image capturing unit which captures images using an imagesensor, comprising: synthesizing a plurality of images; and controllingpredetermined synthesis processing for creating a synthetic image bycausing the synthesizing to synthesize a plurality of images that havebeen obtained by the image capturing unit performing image capturing aplurality of times, wherein, if it is judged that a lens unit that hasbeen replaced after a first image for the synthesis processing wasacquired is a lens unit having an image circle that is different fromthat of a lens unit that was mounted before the replacement, the imagecapturing unit is controlled to not capture a new image for creating thesynthetic image.

The present invention in its fourth aspect provides a control method ofan image capturing apparatus which can interchange a lens unit andincudes an image capturing unit which captures images using an imagesensor, comprising: synthesizing a plurality of images; and controllingpredetermined synthesis processing for creating a synthetic image bycausing the synthesizing to synthesize a plurality of images that havebeen obtained by the image capturing unit performing image capturing aplurality of times, wherein, if it is judged that a lens unit that hasbeen replaced after a first image for the synthesis processing wasacquired is a lens unit that necessitates predetermined imagecorrection, the image capturing unit is controlled to not capture a newimage for creating the synthetic image.

The present invention in its fifth aspect provides a non-transitorycomputer-readable storage medium storing a program which causes, whenread and executed by a computer, the computer to execute steps pf acontrol method of an image capturing apparatus which can interchange alens unit and includes an image capturing unit which captures imagesusing an image sensor, the method comprising: synthesizing a pluralityof images; and controlling predetermined synthesis processing forcreating a synthetic image by causing the synthesizing to synthesize aplurality of images that have been obtained by the image capturing unitperforming image capturing a plurality of times, wherein, if it isjudged that a lens unit that has been replaced after a first image forthe synthesis processing was acquired is a lens unit having an imagecircle that is different from that of a lens unit that was mountedbefore the replacement, the image capturing unit is controlled to notcapture a new image for creating the synthetic image.

The present invention in its sixth aspect provides a non-transitorycomputer-readable storage medium storing a program which causes, whenread and executed by a computer, the computer to execute steps of acontrol method of an image capturing apparatus which can interchange alens unit and incudes an image capturing unit which captures imagesusing an image sensor, the method comprising: synthesizing a pluralityof images; and controlling predetermined synthesis processing forcreating a synthetic image by causing the synthesizing to synthesize aplurality of images that have been obtained by the image capturing unitperforming image capturing a plurality of times, wherein, if it isjudged that a lens unit that has been replaced after a first image forthe synthesis processing was acquired is a lens unit that necessitatespredetermined image correction, the image capturing unit is controlledto not capture a new image for creating the synthetic image.

The present invention in its seventh aspect provides an image capturingapparatus which can interchange a lens unit, includes an image capturingunit, and creates a synthetic image from a plurality of images obtainedby the image capturing unit performing a plurality of times of imagecapturing, the image capturing apparatus comprising: a detecting unitwhich detects remounting of a lens unit; a determining unit whichdetermines, when the detecting unit has detected remounting of a lensunit, a region with respect to which image data of an image sensorincluded in the image capturing unit is extracted based on an imagecircle of the remounted lens unit; an acquiring unit which acquiresimage data with respect to the region determined by the determining unitfrom the image sensor; a judging unit which judges, when synthesizingimage data of interest acquired by the acquiring unit and image data onwhich synthesis is to be performed, whether or not the image size of theimage data of interest is the same as the image size of the image dataon which synthesis is to be performed; and a synthesizing unit whichcreates new synthesis image data on which synthesis is to be performedby synthesizing the image data of interest and the image data on whichsynthesis is to be performed based on a result judged by the judgingunit, wherein the synthesizing unit, if the judging unit has judged thatthe image size of the image data of interest is the same as the imagesize of the image data on which synthesis is to be performed, createsthe new image data on which synthesis is to be performed that has thesame image size as the image on which synthesis is to be performed bysynthesizing the image data of interest and the image data on whichsynthesis is to be performed, and if the judging unit has judged thatthe image size of the image data of interest is different from the imagesize of the image data on which synthesis is to be performed, sets aposition at which an image represented by the image data of interest andan image represented by the image data on which synthesis is to beperformed is to be synthesized, and a size of an image represented bythe synthesized image data, and creates the new image data on whichsynthesis is to be performed based on the setting.

The present invention in its eighth aspect provides a control method ofan image capturing apparatus which can interchange a lens unit, includesan image capturing unit, and creates a synthetic image from a pluralityof images obtained by the image capturing unit performing a plurality oftimes of image capturing, the method comprising: detecting remounting ofa lens unit; determining, when remounting of a lens unit is detected inthe detecting, a region with respect to which image data of an imagesensor included in the image capturing unit is extracted based on animage circle of the remounted lens unit; acquiring image data withrespect to the region determined in the determining from the imagesensor; judging, when synthesizing image data of interest acquired inthe acquiring and image data on which synthesis is to be performed,whether or not the image size of the image data of interest is the sameas the image size of the image data on which synthesis is to beperformed; and creating new image data on which synthesis is to beperformed by synthesizing the image data of interest and the image dataon which synthesis is to be performed based on a result judged in thejudging, wherein in the creating, if the image size of the image data ofinterest is judged as being the same as the image size of the image dataon which synthesis is to be performed in the judging, the new image datais created on which synthesis is to be performed having the same imagesize as the image on which synthesis is to be performed by synthesizingthe image data of interest and the image data on which synthesis is tobe performed, and if the image size of the image data of interest isjudged as being different from the image size of the image data on whichsynthesis is to be performed in the judging, a position is set at whichan image represented by the image data of interest and an imagerepresented by the image data on which synthesis is to be performed isto be synthesized, a size of an image represented by the synthesizedimage data is set, and the new image data on which synthesis is to beperformed is created based on the setting.

The present invention in its ninth aspect provides a non-transitorycomputer-readable storage medium which causes, when read and executed bya computer, the computer to execute step of a control method of an imagecapturing apparatus which can interchange a lens unit, includes an imagecapturing unit, and creates a synthetic image from a plurality of imagesobtained by the image capturing unit performing a plurality of times ofimage capturing, the control method comprising: detecting remounting ofa lens unit; determining, when remounting of a lens unit is detected inthe detecting, a region with respect to which image data of an imagesensor included in the image capturing unit is extracted based on animage circle of the remounted lens unit; acquiring image data withrespect to the region determined in the determining from the imagesensor; judging, when synthesizing image data of interest acquired inthe acquiring and image data on which synthesis is to be performed,whether or not the image size of the image data of interest is the sameas the image size of the image data on which synthesis is to beperformed; and creating new image data on which synthesis is to beperformed by synthesizing the image data of interest and the image dataon which synthesis is to be performed based on a result judged in thejudging, wherein in the creating, if the image size of the image data ofinterest is judged as being the same as the image size of the image dataon which synthesis is to be performed in the judging, the new image datais created on which synthesis is to be performed having the same imagesize as the image on which synthesis is to be performed by synthesizingthe image data of interest and the image data on which synthesis is tobe performed, and if the image size of the image data of interest isjudged as being different from the image size of the image data on whichsynthesis is to be performed in the judging, a position is set at whichan image represented by the image data of interest and an imagerepresented by the image data on which synthesis is to be performed isto be synthesized, a size of an image represented by the synthesizedimage data is set, and the new image data on which synthesis is to beperformed is created based on the setting.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block configuration diagram of an image capturing apparatusaccording to an embodiment.

FIG. 2 is a diagram illustrating one example of a setting menu screenregarding multiple-exposure imaging.

FIGS. 3A to 3C are diagrams illustrating one example of an image to beshot using multiple-exposure imaging.

FIG. 4 is a flowchart illustrating operations of the image capturingapparatus according to the first embodiment.

FIGS. 5A and 5B are diagrams illustrating a difference in image circleaccording to a lens unit.

FIG. 6 is a diagram illustrating one example of a warning messageaccording to the first embodiment.

FIG. 7 is a flowchart illustrating operations of an image capturingapparatus in a second embodiment.

FIG. 8 is a diagram illustrating one example of a warning message in thesecond embodiment.

FIG. 9 is a flowchart illustrating operations of an image capturingapparatus in a third embodiment.

FIG. 10 is a flowchart illustrating operations of an image capturingapparatus in a fourth embodiment.

FIGS. 11A and 11B are diagrams illustrating a relationship between theimage circle and the size of an obtained image.

FIG. 12 is a flowchart illustrating processing when a lens unit isreplaced in a fifth embodiment.

FIG. 13 is a flowchart illustrating processing when multiple-exposureshooting is performed in the fifth embodiment.

FIG. 14 is a diagram illustrating one example of a synthesizing sequenceof image synthesis processing in the fifth embodiment.

FIG. 15 is a diagram illustrating one example of a synthesizing sequenceof image synthesis processing in the fifth embodiment.

FIG. 16 is a diagram illustrating one example of a synthesizing sequenceof image synthesis processing in the fifth embodiment.

FIG. 17 is a diagram illustrating one example of a synthesizing sequenceof image synthesis processing in the fifth embodiment.

FIG. 18 is a flowchart of multiple-synthesis processing in the fifthembodiment.

FIG. 19 is a flowchart illustrating processing for designating asynthesis position in the fifth embodiment.

FIGS. 20A and 20B are diagrams illustrating an example of a GUI whendesignating a synthesis position in the fifth embodiment.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, embodiments of the present invention will be described indetail according to the attached drawings. Note that the configurationsshown in the following embodiments are merely examples, and the presentinvention is not limited to the shown configurations.

FIG. 1 is a block configuration diagram of an image capturing apparatustypified by an interchangeable lens type digital camera in theembodiment. The image capturing apparatus in the embodiment isconstituted by a main-body of the image capturing apparatus 100 and alens unit 180 that can be mounted to/unmounted from the main-body of theimage capturing apparatus 100.

In the main-body of the image capturing apparatus 100, a shutter 120controls the exposure amount of a later-described image capturing device122. The image capturing device 122 includes an image sensor that isconstituted by a CCD or CMOS element that converts an optical image toan electric signal. R, G, and B color filters are arranged on a surfaceof the image sensor in a Bayer arrangement. Therefore, the image dataimmediately after obtaining of an image by the image capturing device122 is constituted by pixel data obtained from pixels in the Bayerarrangement. Also, the image capturing device 122 has an A/D conversionprocessing function. An AF evaluation value detection unit 123calculates an AF evaluation value from contrast information obtainedfrom a digital image signal or a phase difference obtained from parallaximages, and the image capturing device 122 outputs the obtained AFevaluation value to a system control unit 150. Note that the imagesensor in the present embodiment is a so-called full-size sensor.

An image processing unit 124 performs predetermined pixel interpolationprocessing, resizing processing such as reduction, and color conversionprocessing on image data output from the image capturing device 122 orimage data from a memory control unit 115. Also, the image processingunit 124 can acquire information regarding the distance between asubject and the image capturing apparatus. Specifically, the imageprocessing unit 124 acquires the distance to a subject by detecting aphase difference between two given parallax images, and acquiresinformation regarding distance to the subject from the image capturingapparatus for each pixel. Moreover, the image processing unit 124performs predetermined computation processing using image data obtainedby capturing an image, and the system control unit 150 performs exposurecontrol and distance measurement control based on the computationresult. Thus, AE (automatic exposure) processing and EF (automatic flashadjustment) processing of TTL (through-the-lens) type are performed.Also, the image processing unit 124 performs AF (autofocus) processing,and may use an output of the AF evaluation value detection unit 123included in the image capturing device 122 when performing the AFprocessing. Moreover, the image processing unit 124 also performspredetermined computation processing using image data obtained bycapturing an image, and performs AWB (automatic white balance)processing of the TTL type using the obtained computation result. Also,the image processing unit 124 also creates a multiplexed image byperforming processing for synthesizing a plurality of images. Moreover,the image processing unit 124 converts RAW image data in a Bayerarrangement to image data including three components per pixel byperforming debayering processing, and also performs encoding anddecoding processing.

The output data of the image capturing device 122 is written into amemory 132 via the image processing unit 124 and a memory control unit115, or directly written into the memory 132 via the memory control unit115. The memory 132 stores image data that has been acquired by theimage capturing device 122 and subjected to A/D conversion, and imagedata for display in a display unit 128. The memory 132 has a sufficientmemory capacity for storing a predetermined number of still images and amoving image and audio of a predetermined length. Also, the memory 132functions as a memory for image display (video memory). A D/A converter113 converts data for image display stored in the memory 132 to analogsignals and supplies the analog signals to the display unit 128. In thisway, image data for display written into the memory 132 is displayed inthe display unit 128 via the D/A converter 113. The display unit 128performs display on a display such as an LCD in accordance with theanalog signals from the D/A converter 113. The digital signals obtainedby A/D conversion performed by the image capturing device 122 and storedin the memory 132 are converted into analog signals by the D/A converter113, and the analog signals are successively transmitted to the displayunit 128 for display, making it possible to realize an electronicviewfinder that performs through-the-lens image display.

A nonvolatile memory 156 is an electrically erasable and recordablememory, and is a flash memory or the like. Constants and programs, forexample, for operating the system control unit 150 are stored in thenonvolatile memory 156. In this context, “programs” may refer toprograms for executing various flowcharts that will be described later.Also, the nonvolatile memory 156 stores a lens table 156 a in which amodel name of a lens unit that can be connected to the main-body of theimage capturing apparatus 100 is associated with characteristicinformation of the lens unit. The characteristic information of a lensunit includes the size of an image circle.

The system control unit 150 integrally controls the processing units ofthe main-body of the image capturing apparatus 100 by executing programsstored in the above-described nonvolatile memory 156. The reference sign152 indicates a system memory, and is a RAM. Constants and variables foroperating the system control unit 150, programs read out from thenonvolatile memory 156, and the like are deployed in the system memory152.

A system timer 153 is a timer unit for measuring time periods forvarious types of controls and the time of an integrated clock. A modeswitching switch 160, a first shutter switch 162 and a second shutterswitch 163 that are included in a shutter button 161, and the operationunit 170 function as operation members for inputting variousinstructions to operate the system control unit 150. The mode switchingbutton 160 switches the operation mode of the system control unit 150 toany of a still image recording mode, a moving image recording mode, anda reproduction mode. The still image recording mode includes anautomatic shooting mode, an automatic scene determination mode, a manualmode, various types of scene modes in which different settings areconfigured for individual shooting scenes, a program AE mode, a custommode, a multiple-exposure imaging mode, and the like. The mode can bedirectly switched to any of these modes by using the mode switchingbutton 160. Alternatively, it is also possible to switch, using the modeswitching button 160, to the still image recording mode and then toswitch, using another operation member, to any of these modes includedin the still image recording mode. Similarly, the moving image recordingmode may also include a plurality of modes.

When the shutter button 161 provided in the main-body of the imagecapturing apparatus 100 is operated, that is, pressed half-way (theshooting preparation instruction), the first shutter switch 162 isturned on and generates a first shutter switch signal SW1. Uponreceiving the first shutter switch signal SW1, the system control unit150 starts AF (autofocus) processing, AE (automatic exposure)processing, AWB (automatic white balance) processing, EF (automaticflash adjustment) processing, and the like.

When the operation of the shutter button 161 is completed, that is, theshutter button 161 is pressed fully (the shooting instruction), thesecond shutter switch 163 is turned on and generates a second shutterswitch signal SW2. Upon receiving the second shutter switch signal SW2,the system control unit 150 starts a series of shooting processing fromreading out the signal from the image capturing device 122 to writing ofimage data to the recording medium 190.

By selecting various functional icons displayed on the display unit 128,appropriate functions for each situation are assigned to the operationmembers of the operation unit 170, and the operation members thus act asvarious function buttons. Examples of these function buttons include anend button, a back button, an image scrolling button, a jump button, anarrow-down button, and an attribute change button. For example, a menuscreen that enables setting of various settings is displayed on thedisplay unit 128 by pressing the menu button. The user can set varioussettings intuitively by using the menu screen, which is displayed in thedisplay unit 128, four-direction (up, down, left, right) buttons, and aSET button. The menu screen that enables setting of various settingsincludes a screen for selecting, when the multiple-exposure imaging modehas been selected, an image that has been shot in the past and isrecorded in the recording medium 190 as a base image, and themultiple-exposure imaging in which a shot image is synthesized on aselected base image is also possible. Moreover, a screen is alsoincluded for enabling selection as to whether or not the size of animage to be read out as a shot image is cropped to an APS-C sensor size,and if a setting for cropping is selected, shooting can be performed inthe APS-C sensor size even in a case where a lens unit for a full-sizesensor is mounted. This cropping processing will be described laterusing FIGS. 5A and 5B.

A power control unit 141 is constituted by, for example, a batterydetection circuit, a DC/DC converter, a switch circuit for changing overthe block to be supplied with power, and detects whether or not abattery has been inserted, the type of the battery, and the residualcapacity thereof. Moreover, the power control unit 141 controls theDC/DC converter in accordance with the detection results and aninstruction of the system control unit 150, and supplies a necessaryvoltage for a necessary length of time to each of the units includingthe recording medium 190.

A power supply unit 140 is constituted by a primary battery such as analkaline battery or a lithium battery, a secondary battery such as aNiCd battery, a NiMH battery, or a lithium-ion battery, an AC adapter,or the like. The memory medium I/F 118 is an interface with a recordingmedium 190 such as a memory card or a hard disk. The recording medium190 is a memory medium such as a memory card for recording shot images,and is constituted by a semiconductor memory, a magnetic disk, and thelike.

A connector 171 electrically connects the main-body of the imagecapturing apparatus 100 to the lens unit 180. The connector 171exchanges a control signal, a status signal, a data signal, and the likebetween the main-body of the image capturing apparatus 100 and the lensunit 180, and also has a function of supplying currents at variousvoltages. Also, whether or not the lens unit 180 is mounted to theconnector 171 is detected by a lens unmount/mount detection unit 154.

Next, the configuration of the lens unit 180 will be described. Theimaging lens 181 is constituted by a plurality of lenses including afocus lens for performing focus adjustment by moving in an optical axisdirection and a zoom lens for performing zooming. Also, an aperture 182adjusts an amount of light to the image capturing device 122 of themain-body of the image capturing apparatus 100. A lens unit control unit183 controls the entire lens unit 180 in accordance with control signalsfrom the system control unit 150 of the main-body of the image capturingapparatus 100, and controls driving of the focus lens and the zoom lensof the imaging lens 181 and the aperture 182. Also, the lens unitcontrol unit 183 includes a memory for storing constants and variablesfor operations, programs, and the like. Moreover, the lens unit controlunit 183 also includes a nonvolatile memory for storing identificationinformation such as a model name specific to the lens unit 180 and aproduction number thereof, management information, functionalinformation such as a maximum aperture, a minimum aperture, a focallength, an amount of aberration such as distorted aberration, currentand past setting values, and the like. A connector 184 electricallyconnects the lens unit 180 to the main-body of the image capturingapparatus 100. The connector 184 exchanges a control signal, a statussignal, a data signal, and the like between the main-body of the imagecapturing apparatus 100 and the lens unit 180, and also has a functionof receiving currents at various voltages.

Next, basic operations of the multiple-exposure imaging will bedescribed using FIGS. 2 and 3. FIG. 2 is a setting menu screen for themultiple-exposure imaging, and FIG. 3 shows an image to be shot in themultiple-exposure imaging.

When a user presses down the menu button of the operation unit 170, amenu screen for various settings is displayed. When the user selects asetting item for the multiple-exposure imaging by operating thefour-direction buttons of the operation unit 170, a sub-menu screen 200for the multiple-exposure imaging shown in FIG. 2 is displayed in thedisplay unit 128, and the user can change the setting for themultiple-exposure imaging.

The multiple-exposure imaging menu screen 200 includes setting items 201to 204 and a button 205 for selecting a reference image. Among theseitems, the setting item 201 is a setting item for selecting whether ornot the multiple-exposure imaging is to be performed, and one of “Yes”and “No” is selected. If the user exits this menu after selecting “Yes”,and returns to shooting, the multiple-exposure imaging mode is entered.

The setting item 202 is a setting item for selecting the method ofoverlaying exposures when the multiple-exposure imaging is performed,and the user can select one of “addition”, “arithmetic mean”,“comparison (bright)”, and “comparison (dark)”. When “addition” isselected, the luminance of each of a plurality of images to besynthesized are simply added. When “arithmetic mean” is selected, theluminance of each of a plurality of images to be synthesized are addedso as to obtain the luminance that is the average luminance of theimages to be overlaid, which will be described later. When “comparison(bright)” is selected, overlaying is performed such that the luminanceis compared between the images to be synthesized, and brighter portionsare kept. When “comparison (dark)” is selected, overlaying is performedsuch that the luminance is compared between the images to besynthesized, and darker portions are kept.

The setting item 203 is a setting item for selecting the number ofimages to be synthesized when the multiple-exposure imaging isperformed, and the user can designate the number of images to beoverlaid. In the multiple-exposure imaging mode, once the set number ofimages are shot and a synthetic image is recorded, the mode is returnedto a normal snap mode from the multiple-exposure imaging mode.

The setting item 204 is a setting item for selecting the image to berecorded to the recording medium 190 in the multiple-exposure imaging,and the user can select one of “all images” and “only multiplexedimage”. When “all images” is selected, both of the plurality of imagesbased on which overlaying was performed, and the multiple-exposuresynthetic image are recorded in the recording medium 190. When “onlymultiplexed image” is selected, only the multiple-exposure syntheticimage is recorded in the recording medium 190, and the individual imagesbased on which overlaying was performed are not recorded.

When the user presses the button 205, one image file that is alreadystored in the recording medium 190 is to be selected, and amultiple-exposure synthetic image is to be created and recorded. Notethat, if the user sets “5” in the setting item 203, and selects onerecorded image file (RAW image file) by pressing down the button 205,the multiple-exposure synthetic image is created using the imageindicated by the selected image file as the first image, and theremaining four images that will be shot thereafter.

Note that the configuration may be such that two or more image files canbe selected. In this case, if two image files are selected, for example,a multiple-exposure synthetic image may be created from the two imagesindicated by the selected two files and three images that will be shotthereafter.

The reference sign 301 in FIG. 3A indicates a first shot image in themultiple-exposure imaging mode, and the reference sign 302 in FIG. 3Bindicates a second shot image. Also, the reference sign 303 in FIG. 3Cindicates a multiple-exposure synthetic image at the time when thesecond shooting is ended.

First Embodiment

FIG. 4 is a flowchart indicating the processing procedure of the systemcontrol unit 150 in the multiple-exposure imaging mode according to thefirst embodiment. The main-body of the image capturing apparatus 100enters the multiple-exposure imaging mode when the user selects themultiple-exposure imaging mode using the mode switch 160 or selectingthe multiple-exposure imaging in a menu setting, and the processing inFIG. 4 is executed. The processing in each step in FIG. 4 is realized bythe system control unit 150 deploying a predetermined program read outfrom the nonvolatile memory 156 in the system memory 152, andcontrolling the operation and processing of each of the units thatconstitute the main-body of the image capturing apparatus 100. Note thatit is assumed that various settings (such as the number of images to besynthesized and designation of an image file to be synthesized) havebeen already performed by the user, and the set information is stored inthe system memory 152. Note that the configuration may be such that thenumber of images to be synthesized when the previous multiple-exposureimaging is performed is the number of images to be synthesized at thistime unless specifically changed. In this case, the number of images tobe synthesized may be stored in the nonvolatile memory 156.

In step S401, the system control unit 150 acquires the number of imagesto be multiple-synthesized N, which is one piece of setting informationin the multiple-exposure imaging mode, from the system memory 152 or thenonvolatile memory 156. Next, in step S402, the system control unit 150judges whether or not the base image selection has already beenperformed in which the base image is selected from already shot imagesas the first image based on information stored in the system memory 152or the nonvolatile memory 156 in order to start the multiple-exposureimaging. If it is judged that the base image selection has not beenperformed (NO in step S402), the system control unit 150 advances theprocessing to step S403, and substitutes an initial value of “0” for avariable n for counting the number of shootings. On the other hand, ifit is judged that the base image selection has been performed (YES instep S402), the system control unit 150 advances the processing to stepS404, and substitutes an initial value of “1” for the variable n.

In step S405, the system control unit 150 judges whether or notremounting of the lens unit has been performed based on a signal fromthe lens unmount/mount detection unit 154. The remounting of the lensunit includes a case where the mounted lens unit is removed once andthen mounted again, and a case where the mounted lens unit is replacedwith a lens unit that is different from the lens unit that was mounted,for example.

If it is judged that the lens unit has been remounted (YES in stepS405), the system control unit 150 advances the processing to step S406,and if it is judged that the lens unit has not been remounted (NO instep S405), the system control unit 150 advances the processing to stepS409.

In step S406, the system control unit 150 judges whether or not thevariable n is “0”, that is, the base image is not selected and no imageis shot after entering the multiple-exposure imaging mode. In otherwords, the system control unit 150 judges whether or not an image onwhich multiple synthesis is to be performed is present. If it is judgedthat the variable n is “0” (YES in step S406), the system control unit150 advances the processing to step S409. Also, if the variable n is not“0” (NO in step S406), the system control unit 150 advances theprocessing to step S407.

In step S407, the system control unit 150 judges whether or not theimage circle of the newly mounted lens unit is different from the imagecircle of the lens unit that was mounted.

Specifically, the system control unit 150 can communicate with themounted lens unit 180 via the connector 171, and acquire identificationinformation such as a model name. Also, the system control unit 150 canacquire the image circle of the mounted lens unit by searching a lenstable 156 a in the nonvolatile memory 156 using the acquiredidentification information. The system control unit 150 performs thejudgement processing in step S407 by comparing the image circle of thelens unit prior to the lens unit being remounted (which is assumed to bestored in a predetermined address of the nonvolatile memory 156) and theimage circle of the lens unit after being remounted.

Note that the judgement processing in step S407 may be performed basedon whether or not the size of an image to be read out from the imagecapturing device 122 and to be recorded needs to be switched accordingto the mounted lens unit. Moreover, the judgement processing may beperformed based on whether or not the amount of vignetting in theperipheral portion of an image read out from the image capturing device122 that has been detected by scanning the image using the imageprocessing unit 124 or the system control unit 150 is different from theprevious amount of vignetting.

Here, the difference in the image circle will be described using FIGS.5A and 5B. The reference signs 501 and 502 in FIGS. 5A and 5B show amanner in which light that has passed through the lens unit forms animage on an image sensor of the image capturing device 122. Thereference sign 501 shows a case when a lens unit for a full-size sensoris attached, and the reference sign 502 shows a case when a lens unitfor an APS-C size sensor is mounted. As described above, the imagesensor in the present embodiment is a full-size sensor. However, when alens unit for an APS-C size sensor is mounted, light is not incident ona peripheral portion of the image sensor, and “vignetting” occurs aroundthe image. Therefore, when a lens unit for an APS-C size sensor ismounted, the system control unit 150 performs cropping processing inwhich an image of the region indicated by the reference sign 503, shownin the drawing, inside the image sensor is cropped out as a shot imageby controlling the image capturing device 122. As a result of performingthe cropping processing in this way, one image in which the vignettingportion is removed can be obtained, although the size of the imagedecreases.

On the other hand, in the case of the multiple-exposure imaging, sinceimages are synthesized as shown in FIG. 3, when a lens unit whose imagecircle is different is mounted mid-multiple-exposure imaging, if thesynthesis is performed as is, the vignetting portion is alsosynthesized, resulting in an unnatural synthetic image. Also, when thesize of the read-out image is reduced to the APS-C sensor size, that is,a portion on which light is incident is cropped and read out, the sizeof the image differs and synthesis is not possible.

Therefore, in the present embodiment, if it is judged that the imagecircle of the remounted lens unit is the same as the image circle of thepreviously mounted lens unit (NO in step S407), the system control unit150 advances the processing to step S409. Also, if it is judged that theimage circle of the remounted lens unit is different from the imagecircle of the previously mounted lens unit (YES in step S407), thesystem control unit 150 advances the processing to step S408 in order tointerrupt the image capturing processing thereafter.

In step S408, the system control unit 150 displays a message saying thatthe multiple-exposure imaging has ended (FIG. 6) in the display unit 28.Also, the system control unit 150 advances the processing to step S415in order to record the synthetic image obtained by shooting that hasbeen already performed to the recording medium 190 as an image file.

In step S409, the system control unit 150 judges whether or not thesecond shutter switch 163 is turned on (whether or not the secondshutter switch signal SW2 is generated). If it is judged that the secondshutter switch 163 is not turned on (NO in step S409), the systemcontrol unit 150 returns the processing to step S405, and waits for thesecond shutter switch 163 to be turned on. Then, if it is judged thatthe second shutter switch 163 is turned on, the system control unit 150advances the processing to step S410.

In step S410, the system control unit 150 performs capturing in a stillimage frame using the image capturing device 122, stores the obtainedimage data (RAW image data in a Bayer arrangement) in the memory 132,and advances the processing to step S411. In step S411, the systemcontrol unit 150 increments the value of the variable n by “1”, andadvances the processing to step S412.

In step S412, the system control unit 150 judges whether or not thevalue of the variable n is larger than “1”. That is, the system controlunit 150 judges whether or not an image to be processed in the synthesisprocessing is already present. If it is judged that the variable n is“1” or less (NO in step S412), the system control unit 150 returns theprocessing to step S405. Also, if it is judged that the variable n islarger than “1” (YES in step S412), the system control unit 150 advancesthe processing to step S413.

In step S413, the system control unit 150, by controlling the imageprocessing unit 24, synthesizes pieces of image data obtained bycapturing an image in step S410 and stored in the memory 132, generatessynthesized image data, and stores the synthesized image data in thememory 132. If the variable n is larger than “2”, that is, if the imageobtained by capturing in step S410 is the third or later image, thesystem control unit 150 synthesizes the image data obtained by capturingan image with the synthesized image data obtained in the previoussynthesis processing, creates new synthesized image data, and stores thenewly created synthesized image data in the memory 132. Upon ending theprocessing in step S413, the system control unit 150 advances theprocessing to step S414. Note that, in the synthesis processing in stepS413, the synthesis processing is performed with the image data in theBayer arrangement as is.

In step S414, the system control unit 150 judges whether or not thevariable n is equal to “N”, that is whether or not the multiple-exposureimaging is to be ended. If it is judged that the variable n is not equalto “N” (NO in step S414), the system control unit 150 returns theprocessing to step S405. Also, if it is judged that the variable n isequal to “N”, the system control unit 150 advances the processing tostep S415.

In step S415, the system control unit 150, by controlling the imageprocessing unit 124, performs development processing (includingdebayering processing), that is, processing such as white balanceadjustment and pixel interpolation, on the synthesized image datacreated in step S413, converts the obtained image data to YUV data, andwrites the YUV data to the memory 132. Upon finishing the processing instep S415, the system control unit 150 advances the processing to stepS416.

In step S416, the system control unit 150, controlling the imageprocessing unit 24, compression-encodes the synthesized image datasubjected to the development processing that was created in step S415 tostill image data in the JPEG format, writes the still image data in theJPEG format to the memory 132, and advances the processing to step S417.

In step S417, the system control unit 150 records the still image datain the JPEG format that was compression-encoded in step S416 to therecording medium 190 via the memory medium I/F 118, and ends the seriesof multiple-exposure imaging. Note that, if “SAVING SHOT IMAGES” isselected in the setting item 204 in FIG. 2, the shot images that havebeen stored in the memory 132 and used in synthesis (images obtained instep S410) are also recorded in the recording medium 190.

As described above, when remounting of a new lens unit is detected whilemultiple-exposure imaging is being performed, and the image circle ofthe newly mounted lens unit is different from the image circle of thepreviously mounted lens unit, a message is displayed, and themultiple-exposure imaging is ended. Therefore, when a lens unit whoseimage circle is different from that of the previously mounted lens unitis mounted while performing multiple-exposure imaging, an inappropriatemultiplexed image can be prevented from being synthesized. For example,when multiple-exposure imaging is started in a state in which a lensunit for a full-size sensor is mounted, and the lens unit is switched toa lens unit for an APS-C size sensor mid-multiple-exposure imaging, aninappropriate multiplexed image can be prevented from being synthesized.Also, a synthetic image using images that have been obtained by shootingbefore the lens unit was switched, although the number thereof is lessthan N that was initially set, can be stored in the recording medium190.

Second Embodiment

Next, the processing of an image capturing apparatus according to asecond embodiment will be described. The configuration of the imagecapturing apparatus according to the second embodiment is the same asthat of the first embodiment (FIG. 1), and the detailed descriptionthereof will be omitted.

In the first embodiment, when a new lens unit is mounted whilemultiple-exposure imaging is being performed, the lens characteristicacquired by the apparatus is the image circle of the lens, and if theimage circle of the newly mounted lens unit is different from the imagecircle of the previously mounted lens unit, a message is displayed, andthe multiple-exposure imaging is ended. In contrast, in the secondembodiment, when a new lens unit is mounted while multiple-exposureimaging is being performed, and the image circle of the newly mountedlens unit is different from the image circle of the previously mountedlens unit, a warning message is displayed, and if a lens unit having thesame image circle is mounted again, the shooting can be continued. Insuch a way, the apparatus controls creating the synthetic image independence on an acquired lens characteristic (in this case, an imagecircle). With this, even if a lens unit whose image circle is differentfrom that of the previously mounted lens unit is mounted, inappropriatemultiplexed images can be prevented from being synthesized. If a lensunit whose image circle is different from that of the previously mountedlens unit is erroneously mounted, the shooting is not ended, and theseries of multiple-exposure imaging can be continued after anappropriate lens unit is remounted.

FIG. 7 is a flowchart indicating the processing procedure of the systemcontrol unit 150 according to the second embodiment in themultiple-exposure imaging mode. Note that it is assumed that varioussettings (such as the number of images to be synthesized and designationof an image file to be synthesized) regarding the multiple-exposureimaging have already been performed by a user, and the set informationis stored in the system memory 152 or the nonvolatile memory 156.

The processing in steps S701 to S707 and steps S709 to S717 in FIG. 7 isthe same as the processing in steps S401 to S407 and steps S409 to S417in FIG. 4, and therefore the description thereof will be omitted. Theprocessing in steps S721 to S726 in FIG. 7, which are steps after NO isjudged in step S707 (corresponding to the steps after NO is judged instep S407 in FIG. 4), is different from the processing in FIG. 4.

In step S721, the system control unit 150 displays a message (FIG. 8)for prompting a user to switch the lens unit again because the mountedlens unit has an image circle that is different from that of thepreviously mounted lens unit in the display unit 128. Also, the systemcontrol unit 150 displays, in the display unit 128, a menu with whichthe user can select whether or not the synthetic image obtained by theshooting performed before the remounting is to be recorded and themultiple-exposure imaging is to be ended, and whether or not thesynthetic image is to be discarded and the multiple-exposure imaging isto be ended, and advances the processing to step S722.

In step S722, the system control unit 150 judges whether or not the userhas selected the option to end the multiple-exposure imaging with themenu displayed in step S721. If it is judged that the user has notselected the option to end the multiple-exposure imaging (NO in stepS722), the system control unit 150 advances the processing to step S724,and if it is judged that the user has selected the option to end themultiple-exposure imaging (YES in step S722), the system control unit150 advances the processing to step S723.

In step S723, the system control unit 150 judges whether or not the userhas selected the option to record the synthetic image obtained by theshooting performed before the remounting with the menu displayed in stepS721. If it is judged that the user has not selected the option torecord the synthetic image (NO in step S723), the system control unit150 ends the series of multiple-exposure imaging. Also, if it is judgedthat the user has selected the option to record the synthetic image (YESin step S723), the system control unit 150 advances the processing tostep S715.

In step S724, the system control unit 150 judges whether or not the lensunit has been remounted using the lens unmount/mount detection unit 154.If it is judged that the lens unit has been remounted (YES in stepS724), the system control unit 150 advances the processing to step S725,and if it is judged that the lens unit has not been remounted (NO instep S724), the system control unit 150 returns the processing to stepS721 to wait for remounting.

In step S725, the system control unit 150 judges whether or not theimage circle of the lens unit that has been remounted is different fromthe image circle of the lens unit that was used in the immediatelyprevious shooting. If it is judged that the image circle of the lensunit that has been remounted again is the same as the image circle ofthe lens unit that was used in the immediately previous shooting (YES instep S725), the system control unit 150 advances the processing to stepS726. If it is judged that the image circle of the lens unit that hasbeen remounted is different from the image circle of the lens unit thatwas used in the immediately previous shooting (NO in step S725), thesystem control unit 150 returns the processing to step S721.

In step S726, the system control unit 150 hides the warning messagedisplayed in step S721, returns the processing to step S705, andcontinues the series of multiple-exposure imaging.

The processing of the series of multiple-exposure imaging from step S709to step S717 is the same as the processing in steps S409 to S417according to the first embodiment, and therefore the description thereofwill be omitted.

According to the second embodiment as described above, when a lens unitwhose image circle is different from that of the previously mounted lensunit is mounted while multiple-exposure imaging is performed, a warningmessage is displayed, and the series of multiple-exposure imaging can beended according to the selection made by the user. Also, when the userhas not selected the option to end the series of multiple-exposureimaging and remounted a lens unit having the same image circle, theseries of multiple-exposure imaging can be continued. With this, when alens unit whose image circle is different from that of the previouslymounted lens unit is mounted, an inappropriate multiplexed image can beprevented from being synthesized, and if such a lens unit is erroneouslymounted, the shooting is not ended, and the series of multiple-exposureimaging can be continued after another lens unit is mounted. Forexample, when multiple-exposure imaging is started in a state in which alens unit for a full-size sensor is mounted, and the lens unit isswitched to a lens unit for an APS-C size sensor mid-multiple-exposureimaging, an inappropriate multiplexed image can be prevented from beingsynthesized. Moreover, if a lens unit for a full-size sensor isremounted, the series of multiple-exposure imaging can be continued.

Third Embodiment

Next, the processing of an image capturing apparatus according to athird embodiment will be described. The configuration of the imagecapturing apparatus according to the third embodiment is the same asthat of the first embodiment (FIG. 1), and the detailed descriptionthereof will be omitted.

In the first embodiment, when a new lens unit is mounted whilemultiple-exposure imaging is being performed, if the image circle of themounted lens unit is different form the image circle of the previouslymounted lens unit, a message is displayed, and the multiple-exposureimaging is ended.

Similarly, in the third embodiment, when a new lens unit is mountedwhile multiple-exposure imaging is being performed, the apparatusacquires a lens characteristic of whether the lens unit necessitatespredetermined image correction, and if a setting for image correctionsuch as distortion correction is enabled with the newly mounted lensunit, that is, if a lens unit with which image correction must beperformed is mounted, a message is displayed, and the multiple-exposureimaging is ended. In such a way, the apparatus controls creating thesynthetic image in dependence on an acquired lens characteristic (inthis case, an indication that the lens unit necessitates predeterminedimage correction). With this, if a lens unit with which image correctionmust be performed is mounted while multiple-exposure imaging is beingperformed, an unnatural multiplexed image can be prevented from beingcreated as a result of synthesizing an image with respect to which imagecorrection is unnecessary and an image with respect to which imagecorrection needs to be performed.

FIG. 9 is a flowchart illustrating the processing procedure of thesystem control unit 150 in a third embodiment in the multiple-exposureimaging mode. Note that it is assumed that various settings (such as thenumber of images to be synthesized and designation of an image file tobe synthesized) regarding the multiple-exposure imaging have beenalready performed by a user, and the set information is stored in thesystem memory 152 or the nonvolatile memory 156. Also, when themultiple-exposure imaging mode is started, the setting of imagecorrection such as distortion correction is fixed to be disabled, andthe multiple-exposure imaging mode is started after the setting isconfigured as such.

The processing in steps S901 to S905 and steps from S909 to S917 in FIG.9 is the same as the processing in steps S401 to S405 and steps S409 toS417 in FIG. 4, and therefore the description thereof will be omitted.The processing in steps S921 to S923 in FIG. 9, which are steps after NOis judged in step S905 (corresponding to the steps after NO is judged instep S405 in FIG. 4), is different from the processing in FIG. 4.

In step S921, the system control unit 150 judges whether or not thesetting for image correction needs to be enabled with the newly mountedlens unit, that is, whether or not a lens unit with which imagecorrection must be performed has been mounted. Whether or not thesetting for image correction needs to be fixed to be enabled may bejudged based on the lens table 156 a in which information regardingwhether or not correction is needed is stored in association with lensunit identification information (model name). The system control unit150 may perform judgement by receiving identification information fromthe mounted lens unit via the connector 171 and acquiring informationregarding whether or not correction needs to be performed with the lensunit from the lens table 156 a. Also, depending on the situation, thesystem control unit 150 may perform judgement using informationregarding correction function with respect to distorted aberration orthe like of the lens unit 180 that has been acquired from the lens unit180 via the connector 171. Moreover, the system control unit 150 mayperform judgement by scanning an image read out from the image capturingdevice 122 by itself or using the image processing unit 124 and bydetecting the amount of aberration of the image.

If it is judged that the setting for image correction need not to fix tobe enabled with the newly mounted lens unit (NO in step S921), thesystem control unit 150 advances the processing to step S909. If it isjudged that the setting for image correction needs to be enabled withthe newly mounted lens unit (YES in step S921), the system control unit150 advances the processing to step S922.

In step S922, the system control unit 150 displays a message saying thatmultiple-exposure imaging will be ended in the display unit 128, andadvances the processing to step S923. In step S923, the system controlunit 150 judges whether or not the value of the variable n is largerthan “1”, that is, whether or not at least two images have been shot forthe multiple-exposure imaging. If the variable n is larger than “1” (YESin step S923), the system control unit 150 advances the processing tostep S915. If it is judged that the variable n is “1” or less (NO instep S923), the system control unit 150 ends the series ofmultiple-exposure imaging.

The processing from still image shooting in step S909 to ending of theseries of multiple-exposure imaging in step S917 is the same as theprocessing in steps S409 to S417 according to the first embodiment, andtherefore the description thereof will be omitted.

According to the third embodiment as described above, when a new lensunit is mounted while multiple-exposure imaging is being performed, ifthe setting for image correction such as distortion correction is fixedto be enabled with the mounted lens unit, that is, if a lens unit withwhich image correction must be performed is mounted, a message isdisplayed and the multiple-exposure imaging is ended. Accordingly, alens unit with which image correction must be performed is mountedduring multiple-exposure imaging in which the setting for imagecorrection is fixed to be disabled, an inappropriate multiplexed imageis prevented from being synthesized. For example, if the lens unit isswitched to a lens unit with which distorted aberration is large and thesetting for image correction needs to be fixed to be enabled despitebeing low cost while multiple-exposure imaging is being performed, aninappropriate multiplexed image can be prevented from being synthesized.

Fourth Embodiment

Next, the processing of an image capturing apparatus according to afourth embodiment will be described. The configuration of the imagecapturing apparatus according to the fourth embodiment is the same asthat of the first embodiment (FIG. 1), and the detailed descriptionthereof will be omitted.

In the third embodiment, when the setting for image correction such asdistortion correction needs to be enabled with a lens unit that is newlymounted while multiple-exposure imaging is being performed, that is,when a lens unit with which image correction must be performed ismounted, a message is displayed and the multiple-exposure imaging isended. In contrast, in the fourth embodiment, the apparatus acquires alens characteristic of whether the lens unit necessitates predeterminedimage correction, and when a lens unit with which image correction mustbe performed is mounted while multiple-exposure imaging is beingperformed, a warning message is displayed, and if a lens unit with whichimage correction need not be performed is remounted, the shooting can becontinued. In such a way, the apparatus controls creating the syntheticimage in dependence on an acquired lens characteristic (in this case, anindication that the lens unit necessitates predetermined imagecorrection). With this, when a lens unit with which image correctionmust be performed is mounted while multiple-exposure imaging in whichthe setting for image correction is disabled is performed, an unnaturalmultiplexed image can be prevented from being synthesized. Moreover, ifsuch a lens unit is erroneously mounted, the shooting is not ended, andthe series of multiple-exposure imaging can be continued after anotherlens unit is mounted.

FIG. 10 is a flowchart illustrating the processing procedure of a systemcontrol unit 150 in the fourth embodiment in a multiple-exposure imagingmode. Note that it is assumed that various settings (such as the numberof images to be synthesized and designation of an image file to besynthesized) regarding the multiple-exposure imaging have been alreadyperformed by a user, and the set information is stored in the systemmemory 152 or the nonvolatile memory 156. Also, when themultiple-exposure imaging mode is started, the setting of imagecorrection such as distortion correction is fixed to be disabled, andthe multiple-exposure imaging mode is started after the setting isconfigured as such.

The processing in steps S1001 to S1005 and steps S1009 to S1017 is thesame as the processing in steps S401 to S405 and steps S409 to S417 inthe first embodiment, and therefore the description thereof will beomitted.

In step S1021, the system control unit 150 judges whether or not a lensunit with which image correction must be performed has been mounted. Ifit is judged that a lens unit with which image correction need not beperformed is mounted (NO in step S1021), the system control unit 150advances the processing to step S1009. If it is judged that a lens unitwith which image correction needs to be performed is mounted (YES instep S1021), the system control unit 150 advances the processing to stepS1022. In step S1022, the system control unit 150 displays, in thedisplay unit 128, a message for prompting a user to switch the lens unitagain because a lens unit with which image correction must be performedhas been mounted. Also, the system control unit 150 displays, in thedisplay unit 128, a menu with which the user can select processing inwhich the synthetic image obtained by the shooting performed before theremounting is recorded and the multiple-exposure imaging is ended, orprocessing in which the synthetic image is discarded and themultiple-exposure imaging is ended, and advances the processing to stepS1023.

In step S1023, the system control unit 150 judges whether or not theuser has selected the option to end the multiple-exposure imaging on themenu displayed in step S1022. If it is judged that the user has notselected the option to end the multiple-exposure imaging (NO in stepS1023), the system control unit 150 advances the processing to stepS1025. Also, if it is judged that the user has selected the option toend the multiple-exposure imaging (YES in step S1023), the systemcontrol unit 150 advances the processing to step S1024.

In step S1024, the system control unit 150 judges whether or not theuser has selected the option to record the synthetic image obtained bythe shooting performed before the remounting with the menu displayed instep S1022. If it is judged that the user has not selected the option torecord the synthetic image (NO in step S1024), the system control unit150 ends the series of multiple-exposure imaging. Also, if it is judgedthat the user has selected the option to record the synthetic image (YESin step S1024), the system control unit 150 advances the processing tostep S1015.

In step S1025, the system control unit 150 judges whether or not thelens unit has been remounted using the lens unmount/mount detection unit154. If it is judged that the lens unit has been remounted (YES in stepS1025), the system control unit 150 advances the processing to stepS1026, and if it is judged that the lens unit has not been remounted (NOin step S1025), the system control unit 150 returns the processing tostep S1022.

In step S1026, the system control unit 150 judges whether or not thesetting for image correction needs to be enabled with the remounted lensunit, that is, whether or not a lens unit with which image correctionmust be performed has been mounted. If it is judged that the setting forimage correction need not be enabled with the remounted lens unit (NO instep S1026), the system control unit 150 advances the processing to stepS1027. If it is judged that the setting for image correction needs to beenabled with the remounted lens unit (YES in step S1026), the systemcontrol unit 150 returns the processing to step S1022.

In step S1027, the system control unit 150 hides the warning messagedisplayed in step S1022, returns the processing to step S1005, andcontinues the series of multiple-exposure imaging.

The processing of the series of multiple-exposure imaging from stepS1009 to step S1017 is the same as the processing in steps S409 to S417according to the first embodiment, and therefore the description thereofwill be omitted.

According to the fourth embodiment as described above, when the settingfor image correction needs to be fixed to be enabled with the lens unitthat has been mounted while multiple-exposure imaging is beingperformed, that is, a lens unit with which image correction must beperformed has been mounted, a warning message is displayed, and theseries of multiple-exposure imaging can be ended according to theselection made by the user. Also, when the user has not selected theoption to end the series of multiple-exposure imaging and remounted alens unit with which image correction need not be performed isremounted, the series of multiple-exposure imaging can be continued.With this, when a lens unit with which image correction must beperformed is mounted while multiple-exposure imaging in which thesetting for image correction is fixed to be disabled is being performed,an inappropriate multiplexed image can be prevented from beingsynthesized. Also, if such a lens unit is erroneously mounted, theshooting is not ended, and the series of multiple-exposure imaging canbe continued after another lens unit is mounted.

Fifth Embodiment

Next, a multiple-exposure imaging mode in a fifth embodiment will bedescribed.

First, the relationship between an image sensor mounted in the imagecapturing device 122 and an image circle of a mounted lens unit will bedescribed with reference to FIGS. 11A and 11B.

As shown in FIG. 11A, if the image circle is larger than the imagesensor surface (in the case of a full size lens unit), a captured image(RAW image data in a Bayer arrangement) using the entirety of the imagesensor can be obtained.

On the other hand, as shown in FIG. 11B, in the case of an APS-C lensunit, the size of the image circle is smaller than the size of the imagesensor. Therefore, a rectangular range 1100 that is inscribed insidethis image circle must be set as the image capturing range. That is,when the size of the image circle is smaller than the size of the imagesensor, as shown in FIG. 11B, the size of the captured image needs to bereduced to be smaller than the size of the captured image shown in FIG.11A.

The system control unit 150 receives identification information (such asmodel name) from a mounted lens unit 180 at a timing at which mountingof the lens unit 180 is detected based on a signal from the lensunmount/mount detection unit 154, or at a timing at which initializationprocessing after power-on is performed. Also, the system control unit150 searches the lens table 156 a using the received identificationinformation as the key, and acquires the image circle of the mountedlens unit. Then, the system control unit 150 judges which of the statesin FIGS. 11A and 11B is realized based on the acquired image circle. Inthe case of the state in FIG. 11A, the system control unit 150 sets theimage processing unit 124 such that a captured image is acquired usingthe entire region of the image sensor. Also, in the case of the state inFIG. 11B, the system control unit 150 sets the image processing unit 124such that an image inside the region 1100 inside the image sensor isacquired as the captured image.

FIG. 12 is a flowchart illustrating processing of the system controlunit 150 when a detection signal indicating that a lens unit has beenmounted is received from the lens unmount/mount detection unit 154.

In step S1201, the system control unit 150 communicates with the lensunit control unit 183 via the connector 171. In step S1202, the systemcontrol unit 150 acquires identification information of the lens unitsuch as a model name. Also, in step S1203, the system control unit 150acquires the image circle of the mounted lens unit by referring to thelens table 156 a, and judges which of the image circle and the imagesensor included in the image capturing device 122 is larger.

In step S1203, if it is judged that the image circle is larger than theimage sensor, the system control unit 150 advances the processing tostep S1204. In step S1204, system control unit 150 judges that the RAWimage size in shooting is “large”. On the other hand, if it is judgedthat the image circle is smaller than the size of the image sensor, thesystem control unit 150 advances the processing to step S1205. In stepS1205, the system control unit 150 judges that the RAW image size inshooting is “small”. Also, in step S1206, the system control unit 150sets the size of the RAW image in a Bayer arrangement when acquiring theRAW image from the image capturing device 122 (either of the entireregion of the image sensor or the region 1100 in the embodiment) to theimage processing unit 124 based on the determined size.

FIG. 13 is a flowchart illustrating the flow of multiple-synthesisprocessing in the image processing unit 124, which is the feature of thefifth embodiment. When a user selects a multiple-exposure imaging modeusing the mode switch 160 or selects multiple-exposure imaging in themenu setting, the system control unit 150 enters the multiple-exposureimaging mode and performs the processing of the multiple-exposureimaging in accordance with the flowchart shown in FIG. 13. Note that theprocessing in each step in FIG. 13 is realized by the system controlunit 150 deploying a predetermined program read out from the nonvolatilememory 156 in the system memory 152, executing the program, andcontrolling the operation and processing of the units that constitutethe main-body of the image capturing apparatus 100. Note that it isassumed that various settings (such as the number of images to besynthesized and designation of an image file to be synthesized) havebeen already performed by the user, and the set information is stored inthe system memory 152. Note that the configuration may be such that thenumber of images to be synthesized when the previous multiple-exposureimaging was performed is the number of images to be synthesized at thistime unless specifically changed. In this case, the number of images tobe synthesized may be stored in the nonvolatile memory 156.

In step S1301, the system control unit 150 acquires the number of imagesto be multiple-synthesized N in the multiple-exposure imaging mode fromthe system memory 152 or the nonvolatile memory 156. Next, in stepS1302, the system control unit 150 judges whether or not the user hasmade a setting to use an image file recorded in the recording medium 190as a first base image to be used in the multiple-exposure imaging. If itis judged that the base image has not been selected (NO in step S1302),the system control unit 150 advances the processing to step S1303, andif it is judged that the base image has been selected (YES in stepS1302), the system control unit 150 advances the processing to stepS1304.

In step S1303, the system control unit 150 initializes a variable n,which indicates the number of shot images, to “0”, and advances theprocessing to step S1305. Also, in step S1304, the system control unit150 initializes the variable n to “1”, and advances the processing tostep S1305.

In step S1305, the system control unit 150 judges whether or not a lensunit has been mounted anew (whether or not remounted) using the lensunmount/mount detection unit 154. The remounting of the lens unitincludes a case where the mounted lens unit is removed once and thenmounted again, and a case where the mounted lens unit is replaced with alens unit that is different from the mounted lens unit, for example. Ifit is judged that a lens unit has been mounted anew (YES in step S1305),the system control unit 150 advances the processing to step S1306, andif it is judged that the lens unit is the same as that used when theprevious shooting was performed, that is, unmounting/mounting has notbeen performed (NO in step S1305), the system control unit 150 advancesthe processing to step S1308.

In step S1306, the system control unit 150 judges whether or not theimage circle of the currently mounted lens unit is different from theimage circle of the lens unit that was used when the image data on whichsynthesis is to be performed was obtained.

Specifically, the system control unit 150 can communicate with themounted lens unit 180 via the connector 171, and acquire identificationinformation such as a model name. Also, the system control unit 150 canacquire the a lens characteristic such as an image circle of the mountedlens unit by searching a lens table 156 a in the nonvolatile memory 156using the acquired identification information. The system control unit150 performs the judgement processing in step S1306 by comparing theimage circle of the lens unit that was used when shooting was performedprior to the lens unit being remounted (which is assumed to be stored ina predetermined address of the nonvolatile memory 156) with the imagecircle of the lens unit after being remounted.

Note that, if the configuration is such that the size of an image to beread out from the image capturing device 122 and to be recorded isswitched according to the mounted lens unit, the judgement processingmay be performed based on the size of an image read out from the imagecapturing device 122. Also, the judgement processing may be performed byperforming image processing on an image read out from the imagecapturing device 122 using the image processing unit 124 and bydetecting the amount of vignetting in a peripheral portion of the image.

If it is judged that the image circle of the currently mounted lens unitis different from the image circle of the lens unit that was used whenthe image data on which synthesis is to be performed was obtained (YESin step S1306), the system control unit 150 advances the processing tostep S1307, and if not (NO in step S1306), the system control unit 150advances the processing to step S1308.

In step S1307, the system control unit 150 performs processing fordesignating a synthesis position (the details will be described later),and advances the processing to step S1308.

In step S1308, the system control unit 150 judges whether or not thesecond shutter switch 163 is turned on (whether or not the secondshutter switch signal SW2 is generated). If it is judged that the secondshutter switch 163 is not turned on (NO in step S1308), the systemcontrol unit 150 returns the processing to step S1305, and if it isjudged that the second shutter switch 163 is turned on (YES in stepS1308), the system control unit 150 advances the processing to stepS1309.

In step S1309, the system control unit 150 acquires image data bycausing the image capturing device 122 to perform capturing in a stillimage frame, and stores the image data to the memory 132. Also, in stepS1310, the system control unit 150 increments the value of the variablen by “1”. In step S1311, the system control unit 150 judges whether ornot the value of variable n is larger than 1, that is, whether or notthe captured image is a second or later image, and image data on whichsynthesis processing is to be performed is already present. If it isjudged that the variable n is “1” or less (NO in step S1311), the systemcontrol unit 150 returns the processing to step S1305, and if it isjudged that the variable n is larger than “1” (YES in step S1311), thesystem control unit 150 advances the processing to step S1312.

In step S1312, the system control unit 150, by controlling the imageprocessing unit 124, creates new synthesized image data by synthesizingimage data of interest that has been obtained by the capturing of animage in step S1309 and stored in the memory 132 and the image data onwhich synthesis is to be performed, and stores the new synthesized imagedata in the memory 132.

Here, if the value of the variable n is “2”, the image of interestobtained in the latest step S1309 is the second image. Therefore, thesystem control unit 150 stores the synthetic image obtained bysynthesizing the second image of interest and the first image to thememory 132. Also, if the value of the variable n is larger than “2”, theimage of interest is a third or later image. Therefore, in this case,the system control unit 150 creates a new synthetic image bysynthesizing the image of interest obtained in the latest step S1309 andthe previous synthetic image, and stores the new synthetic image to thememory 132.

In step S1313, the system control unit 150 compares the value of thevariable n with “N”. That is, the system control unit 150 judges whetheror not the number of images in the multiple-exposure imaging reaches theset number. If it is judged that the value of the variable n is lessthan “N” (NO in step S1313), the system control unit 150 returns theprocessing to step S1305. Also, if it is judged that the value of thevariable n is “N” (YES in step S1313), the system control unit 150advances the processing to step S1314.

In step S1314, the system control unit 150, by controlling the imageprocessing unit 124, performs development processing, that is,processing such as white balance adjustment and pixel interpolation, onthe synthesized image data created in step S1312, converts the obtainedimage data to YUV data, and writes the YUV data to the memory 132. Also,in step S1315, the system control unit 150, by controlling the imageprocessing unit 124, compression-encodes the synthesized image datasubjected to the development processing that was created in step S1314to still image data in the JPEG format, and writes the still image datain the JPEG format to the memory 132. Also, in step S1316, the systemcontrol unit 150 records the still image data in the JPEG format thatwas compression-encoded in step S1315 to the recording medium 190 viathe memory medium I/F 118, and ends the series of multiple-exposureimaging.

Next, the processing for designating a synthesis position in step S1307will be described with reference to FIG. 19.

In step S1901, the system control unit 150 judges whether the imagecircles when two images on which the synthesis processing is to beperformed were shot are the same. If it is judged that the image circlesare the same, or if the image data on which synthesis is to be performedand image data to be synthesized are not prepared (the number of shotimages is less than 2), the system control unit 150 does not perform theprocessing for designating a synthesis position, and ends thisprocessing. Also, if it is judged that the image circles when the twoimages on which the synthesis processing is to be performed were shotare not the same (NO in step S1901), the system control unit 150advances the processing to step S1902.

In step S1902, the system control unit 150, by controlling the displayunit 128, displays a screen for designating a synthesis position, andaccepts a user operation via the operation unit 170.

In order to simplify the description, a case where the image circle ofimage data on which synthesis is to be performed (first image data, orsynthesized image data obtained in the synthesis processing in thelatest step S1312) is smaller than the image circle of the currentlymounted lens unit will be described.

FIG. 20A shows an initial screen of the screen for designating asynthesis position in such a state. The system control unit 150 displaysa live video obtained through the currently mounted lens unit, in theentire region of a region 2001 in the display unit 128. Then, the systemcontrol unit 150 displays an image on which synthesis is to be performedin a region 2002, which can be moved according to the user operation.When the user sees the screen, a synthetic image that has been obtainedby the latest synthesis processing is movably displayed in an image of alive video. Also, the system control unit 150 moves the region 2002inside the region 2001 according to the user operation made on theoperation unit 170. Upon receiving an instruction from the user todetermine the position via the operation unit 170, the system controlunit 150 determines the position of the region 2002. That is, the systemcontrol unit 150 determines the position of the image on which synthesisis to be performed relative to the image to be synthesized.

Note that, if the image circle of the image on which synthesis is to beperformed is larger than the image circle of the currently mounted lensunit, the system control unit 150 displays the image on which synthesisis to be performed in the entire region of the region 2001, and displaysthe live video in the region 2002. Then, the user performs designationof the position of the region 2002 and determination instruction,similarly to the operations described above. The different point for theuser is that the position of the live video is to be determined insteadof the image on which synthesis is to be performed.

In step S1903, the system control unit 150 prompts the user to select,as the final synthesis size, one of the size (full size) of an image ofthe image circle indicated by the region 2001 and the size of an imagewhose image circle is smaller, which is indicated by the region 2002.Then, the system control unit 150 waits for receiving the selectioninstruction from the user via the operation unit 170.

In step S1904, the system control unit 150 judges whether or not thefinal synthesis size selected by the user is the full size. If it isjudged that the final synthesis size selected by the user is the fullsize (YES in step S1904), the system control unit 150 advances theprocessing to step S1906. On the other hand, if it is judged that thefinal synthesis size selected by the user is not the full size (NO instep S1904), the system control unit 150 advances the processing to stepS1905.

In step S1905, the system control unit 150 keeps the display state ofthe display unit 128 to be the state shown in FIG. 20A in a period fromnow on until the second shutter switch 163 is turned on.

In step S1906, the system control unit 150 displays a live syntheticimage of the region whose image circle is smaller in the entire regionof the region 2001 of the display unit 128 in a period from now on untilthe second shutter switch 163 is turned on. That is, as shown in FIG.20B, even if the currently mounted lens unit is a lens unit for a fullsize image sensor, the system control unit 150 displays, in the region2001, a synthetic image obtained by cropping, similarly to the case ofthe smaller image circle. If stated simply, it is more understandable toconsider that the image in the region 2002 in FIG. 20A is enlarged tothe size of the entire region of the region 2001, and is displayedtherein.

In step S1907, the system control unit 150 configures the setting of theimage processing unit 124 such that synthesizing is performed, in thefinal synthesis size, using the information indicating the synthesisposition determined as described above. As a result, the system controlunit 150 causes the image processing unit 124 to perform synthesisprocessing under the setting described above in step S1312 in theflowchart shown in FIG. 13. Of course, the synthetic image obtained instep S1312 is one of a full-size image and an APS-C size image.

Next, examples of processing for synthesizing four images through theprocessing for designating a synthesis position in step S1307, which isperformed by the image processing unit 124, will be described withreference to FIGS. 14 to 17.

FIG. 14 shows an example in which first and second images are capturedusing a lens unit whose image circle is small (APS lens unit), third andfourth images are captured using a lens unit whose image circle is large(full-size lens unit), and the size of the ultimate synthetic imagedesignated by the user is a size of an image whose image circle issmall.

The image processing unit 124 obtains images 1401 and 1402 by croppingregions corresponding to the APS image circle from the full-size images,when capturing the first and second images. Because the images 1401 and1402 are images (cropped images) that have been shot using a lens unithaving a small image circle, designation of the synthesis position bythe user is not performed. The image processing unit 124 creates asynthetic image 1411 (cropped image) by simply synthesizing these twoimages 1401 and 1402. The third image was shot using a lens unit havinga large image circle, and the user has selected a smaller image size asthe size of the ultimate synthetic image. Therefore, the imageprocessing unit 124 acquires an image obtained by cropping the full-sizeimage as a third image 1403. Then, the image processing unit 124 createsa new synthetic image 1412 (cropped image) by synthesizing the syntheticimage 1411 and the image 1403. Note that, here, the position of theimage 1403 to be cropped from the full-size image obtained in the thirdcapturing is designated by the user. Also, the fourth image was shotusing the lens unit having a large image circle, and the user hasselected a smaller image size as the size of the ultimate syntheticimage. Therefore, the image processing unit 124 acquires an imageobtained by cropping the full-size image as a fourth image 1404. Then,the image processing unit 124 creates an ultimate synthetic image 1413(cropped image) by synthesizing the synthetic image 1412 and the image1404. Note that, here, the position of the image 1404 to be cropped fromthe full-size image obtained in the fourth capturing is also designatedby the user.

FIG. 15 shows an example in which first and second images are capturedusing a lens unit whose image circle is large (full-size lens unit),third and fourth images are captured using a lens unit whose imagecircle is small (APS lens unit), and the size of the ultimate syntheticimage designated by the user is the size of an image whose image circleis small.

Since images 1501 and 1502 are pieces of full-size image data obtainedby performing capturing using the lens unit having a large image circle,designation of the synthesis position by the user is not performed whensynthesizing these images. The image processing unit 124 creates asynthetic image 1511 (full-size image) by simply synthesizing these twoimages 1501 and 1502. Since the third image is captured by using a lensunit having a small image circle, the image processing unit 124 createsan image 1503 by cropping the full-size image obtained by the imagecapturing device 122. The user has selected a smaller image size as thesize of the ultimate synthetic image. Therefore, the image processingunit 124 creates a synthetic image 1512 (cropped image) by synthesizingthe synthetic image 1511 (full-size image) and the image 1503 (croppedimage). Note that the synthesis position of the image 1503 on thesynthetic image 1512 is designated by the user. Also, since the fourthimage is captured by using a lens unit having a small image circle, theimage processing unit 124 creates an image 1504 by cropping thefull-size image obtained by the image capturing device 122. The user hasselected a smaller image size as the size of the ultimate syntheticimage. Therefore, the image processing unit 124 creates an ultimatesynthetic image 1513 (cropped image) by synthesizing the synthetic image1512 (cropped image) and the fourth image 1504 (cropped image). Notethat, since the synthetic image 1512 and the image 1504 are croppedimages, position designation by the user is not performed.

FIG. 16 shows an example in which first and second images are capturedusing a lens unit whose image circle is small (APS lens unit), third andfourth images are captured using a lens unit whose image circle is large(full-size lens unit), and the size of the ultimate synthetic imagedesignated by the user is the size of an image whose image circle islarge.

The image processing unit 124 obtains images 1601 and 1602 by croppingregions corresponding to the APS image circle from the full-size images,when capturing the first and second images. Because the images 1601 and1602 are images (cropped images) that have been shot using a lens unithaving a small image circle, designation of the synthesis position bythe user is not performed. The image processing unit 124 creates asynthetic image 1611 (cropped image) by simply synthesizing these twoimages 1601 and 1602. The third image was shot using a lens unit havinga large image circle, and the user has selected a larger image size asthe size of the ultimate synthetic image. Therefore, the imageprocessing unit 124 obtains a full-size synthetic image 1612 bysynthesizing the synthetic image 1611 (cropped image) and the full-sizeimage 1603. Note that, here, the synthesis position of the syntheticimage 1611 on the full-size image 1603 obtained in the third capturingis designated by the user. The fourth image was shot using a lens unithaving a large image circle, and the user has selected a larger imagesize as the size of the ultimate synthetic image. Therefore, the imageprocessing unit 124 creates an ultimate synthetic image 1613 (full-sizeimage) by synthesizing the full-size synthetic image 1612 and the fourthfull-size image 1604. Note that, since the synthetic image 1612 and theimage 1604 are full-size images, position designation by the user is notperformed.

FIG. 17 shows an example in which first and second images are capturedusing a lens unit whose image circle is large (full-size lens unit),third and fourth images are captured using a lens unit whose imagecircle is small (APS lens unit), and the size of the ultimate syntheticimage designated by the user is the size of an image whose image circleis large.

Since images 1701 and 1702 are pieces of full-size image data obtainedby performing capturing using a lens unit having a large image circle,designation of the synthesis position by the user is not performed whensynthesizing these images. The image processing unit 124 creates asynthetic image 1711 (full-size image) by simply synthesizing these twoimages 1701 and 1702. Since the third image was captured by using a lensunit having a small image circle, the image processing unit 124 createsan image 1703 by cropping the full-size image obtained by the imagecapturing device 122. The user has selected a larger image size as thesize of the ultimate synthetic image. Therefore, the image processingunit 124 creates a synthetic image 1712 (full-size image) bysynthesizing the synthetic image 1711 (full-size image) and the image1703 (cropped image). Note that the synthesis position of the image 1703on the synthetic image 1711 is designated by the user. Also, since thefourth image was captured by using a lens unit having a small imagecircle, the image processing unit 124 creates an image 1704 by croppingthe full-size image obtained by the image capturing device 122. The userhas selected a larger image size as the size of the ultimate syntheticimage. Therefore, the image processing unit 124 creates an ultimatesynthetic image 1713 (full-size image) by synthesizing the syntheticimage 1712 (full-size image) and the fourth image 1704 (cropped image).Note that the synthesis position of the image 1704 on the syntheticimage 1712 is designated by the user.

Next, the multiple-synthesis processing of the image processing unit 124in the embodiment will be described with reference to FIG. 18. Thismultiple-synthesis processing is the processing in step S1312 in FIG.13.

In step S1801, the image processing unit 124 judges whether or not thesize of the RAW shot image obtained in step S1309 matches the size ofthe image on which synthesis is to be performed. If it is judged thatthe sizes are the same (YES in step S1801), the image processing unit124 advances the processing to step S1807, and if not (NO in stepS1801), the image processing unit 124 advances the processing to stepS1802.

In step S1802, the image processing unit 124 judges whether or not thecomparison synthesis has been selected as the synthesis method ofmultiple-exposure imaging in the menu setting. If it is judged that thecomparison synthesis has been selected as the synthesis method ofmultiple-exposure imaging (YES in step S1802), the image processing unit124 advances the processing to step S1803, and if it is judged that thecomparison synthesis has not been selected (NO in step S1802), the imageprocessing unit 124 advances the processing to step S1804.

In step S1803, the image processing unit 124 configures the setting suchthat the pixel values of the full-size image outside the region of thecropped RAW image are kept at 100%, and advances the processing to stepS1807.

In step S1804, the image processing unit 124 judges which of bright anddark has been selected in the comparison synthesis in the menu setting.If it is judged that the synthesis of relatively bright portions hasbeen selected (YES in step S1804), the image processing unit 124advances the processing to step S1806, and if it is judged that thesynthesis of relatively bright portions has been selected (NO in stepS1804), the image processing unit 124 advances the processing to stepS1805.

In step S1806, the image processing unit 124 configures the setting suchthat the region outside the region of the cropped RAW image is a blackregion (all components are at minimum luminance), and advances theprocessing to step S1807. In step S1805, the image processing unit 124configures the setting such that the region outside the region of thecropped RAW image is a white region (all components are at maximumluminance), and advances the processing to step S1807.

In step S1807, the image processing unit 124 performs the synthesisprocessing according to the condition set in steps S1803, S1805, andS1806 at a designated synthesis position if the synthesis position hasbeen designated by the user, and stores the synthesized image data tothe memory 132.

According to the present embodiment as described above, themultiple-exposure imaging can be continued even if, in a camera in whichthe lens unit can be unmounted and mounted, the lens unit is switched toa lens unit having a different image circle size, and moreover, thesynthesis can be performed at a position in the image desired by a user.

Note that, in the embodiment described above, at least a live image isdisplayed in the display unit 128 when the multiple-exposure imaging isperformed, but the configuration may be such that a synthetic image oftwo images may be displayed only when positioning is performed insynthesis processing. Also, in the embodiment described above, a userselects the size of the ultimate synthetic image every time images ofdifferent image circles are synthesized, but the configuration may besuch that a final synthesis size is provided as a setting item formultiple-exposure imaging in FIG. 12, and the final synthesis size isdetermined when this setting is configured.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2018-120040, filed Jun. 25, 2018, and Japanese Patent Application No.2018-134477, filed Jul. 17, 2018 which are hereby incorporated byreference herein in their entirety.

What is claimed is:
 1. An image capturing apparatus which has a firstimage sensor and can interchange a lens unit, comprising: at least oneprocessor and/or circuit configured to function as a control unit,wherein the control unit performs a predetermined synthesizing processto perform a plurality of image captures using the first image sensorand to create a synthetic image from a plurality of images obtained bythe plurality of captures, wherein, in the predetermined synthesizingprocess, in a case where, a first lens unit for the first image sensoris remounted after a first image capture is performed to create thesynthetic image in a state in which a second lens unit for a secondimage sensor whose size is smaller than the size of the first imagesensor is mounted, the control unit continues the predeterminedsynthesizing process which had been performed before the remount andenables capture of a second image, and controls a display unit todisplay a live video which is obtained via the remounted first lens unitand for which a region of an image corresponding to the size of thesecond image sensor is cropped from a region of an image correspondingto the size of the first image sensor.
 2. The apparatus according claim1, wherein, the control unit controls the display unit to synthesize thelive video and a first image obtained by the first capturing and todisplay the synthesized live video.
 3. The apparatus according to claim1, wherein the control unit enables a second image capture while thelive video is displayed, wherein the control unit, based on a firstimage obtained by the first image capture and a second image obtained bythe second image capture, creates the synthetic image in which a regioncorresponding to the size of the second image sensor is cropped out of aregion of an image corresponding to the first image sensor, and recordsthe created synthetic image as a still image file to a recording medium.4. The apparatus according to claim 1, wherein the first lens unit forthe first image sensor has an image circle larger than that of thesecond lens unit for the second image sensor.
 5. The apparatus accordingto claim 1, wherein the first image sensor is a full-size sensor and thesecond image sensor is an APS-C sensor.
 6. The apparatus according toclaim 1, wherein the predetermined synthesizing process is a process ofobtaining a synthetic image by synthesizing a plurality of imagesobtained in multiple-exposure photography.
 7. The apparatus according toclaim 1, wherein each of images to be synthesized to create thesynthetic image is obtained in response to a user's operation forcapturing.
 8. The apparatus according to claim 1, wherein a position ofthe region of the image corresponding to the size of the second imagesensor to be cropped out of the region of image corresponding to thefirst image sensor can be set based on a user operation.
 9. A method ofcontrolling an image capturing apparatus which has a first image sensorand can interchange a lens unit, the method comprising: performing apredetermined synthesizing process to capture a plurality of image usingthe first image sensor, and to create a synthetic image from a pluralityof images obtained by the plurality of image captures, wherein, in thepredetermined synthesizing process, in a case where a first lens unitfor the first image sensor is remounted after a first image capture isperformed to create the synthetic image in a state in which a secondlens unit for a second image sensor whose size is smaller than the sizeof the first image sensor is mounted, continuing the predeterminedsynthesizing process which had been performed before the first lens unitis remounted; and controlling a display unit to display a live videowhich is obtained via the remounted first lens unit and for which aregion of an image corresponding to the size of the second image sensoris cropped from a region of an image corresponding to the size of thefirst image sensor.
 10. A non-transitory computer-readable storagemedium storing a program which causing an image capturing apparatuswhich has a first image sensor and can interchange a lens unit, toexecute a method comprising: performing a predetermined synthesizingprocess to capture a plurality of image using the first image sensor andto create a synthetic image from a plurality of images obtained by theplurality of image captures, wherein, in the predetermined synthesizingprocess, in a case where a first lens unit for the first image sensor isremounted after a first image capture is performed to create thesynthetic image in a state in which a second lens unit for a secondimage sensor whose size is smaller than the size of the first imagesensor is mounted, continuing the predetermined synthesizing processwhich had been performed before the first lens unit is remounted; andcontrolling a display unit to display a live video which is obtained viathe remounted first lens unit and for which a region of an imagecorresponding to the size of the second image sensor is cropped from aregion of an image corresponding to the size of the first image sensor.